[Bug] [linux] [opengl] Use RTLD_LOCAL to prevent LLVM symbol conflict with GLX #1326

archibate · 2020-06-25T10:36:34Z

Related issue = close #958 close #1100 close #1106 close #1113 close #1325

Specifying RTLD_GLOBAL will expose Taichi-customized LLVM symbols to other modules that may depends on other LLVM versions, e.g. GLX depends on LLVM and cause the issues above.

@Eydcao @TroyZhai This issue is finally solved systematically!!! Please confirm that if this PR works when you have time :)

… with GLX

codecov · 2020-06-25T11:07:45Z

Codecov Report

Merging #1326 into master will decrease coverage by 0.46%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1326      +/-   ##
==========================================
- Coverage   85.35%   84.89%   -0.47%     
==========================================
  Files          19       19              
  Lines        3374     3237     -137     
  Branches      630      630              
==========================================
- Hits         2880     2748     -132     
+ Misses        362      357       -5     
  Partials      132      132

Impacted Files	Coverage Δ
python/taichi/lang/expr.py	`89.24% <100.00%> (-0.52%)`	⬇️
python/taichi/lang/ast_checker.py	`70.58% <0.00%> (-1.64%)`	⬇️
python/taichi/lang/ops.py	`91.32% <0.00%> (-1.52%)`	⬇️
python/taichi/lang/linalg.py	`89.33% <0.00%> (-0.67%)`	⬇️
python/taichi/lang/matrix.py	`90.03% <0.00%> (-0.65%)`	⬇️
python/taichi/lang/meta.py	`61.90% <0.00%> (-0.60%)`	⬇️
python/taichi/lang/impl.py	`89.76% <0.00%> (-0.30%)`	⬇️
python/taichi/lang/__init__.py	`75.07% <0.00%> (-0.29%)`	⬇️
python/taichi/lang/snode.py	`85.86% <0.00%> (-0.16%)`	⬇️
... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 13462ff...3aff448. Read the comment docs.

k-ye

LGTM (with no way to verify..)!

masahi · 2020-06-25T12:16:57Z

This fixed the same error on my end with opengl enabled. Thanks!

yuanming-hu

I don't clearly remember why I added RTLD_GLOBAL here, but let's see if everything works :-) Thanks.

Eydcao · 2020-06-26T02:51:58Z

LGTM (with no way to verify..)!

I could verify this on my end, just a sec.

Eydcao · 2020-06-26T04:21:05Z

Unfortunately, sth is still going wrong on my end. Please see the log below

[Taichi] mode=development
[Taichi] preparing sandbox at /tmp/taichi-qe4n7sb9
[Taichi] <dev mode>, llvm 8.0.1, commit 3aff448e, python 3.8.2
[Taichi] Starting on arch=opengl
[E 06/26/20 12:19:59.619] [opengl_api.cpp:link@168] [glsl] error while linking program:
error: Too many compute shader storage blocks (11/8)



***********************************
* Taichi Compiler Stack Traceback *
***********************************
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::Logger::error(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::opengl::GLProgram::link() const
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::opengl::CompiledKernel::CompiledKernel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, taichi::lang::opengl::KernelParallelAttrib const&)
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::opengl::CompiledProgram::Impl::add(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, taichi::lang::opengl::KernelParallelAttrib&&)
/tmp/taichi-qe4n7sb9/taichi_core.so(+0x9ea906) [0x7ff0eeb0d906]
/tmp/taichi-qe4n7sb9/taichi_core.so(+0x9dfc7b) [0x7ff0eeb02c7b]
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::opengl::OpenglCodeGen::gen()
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::opengl::OpenglCodeGen::compile(taichi::lang::Program&, taichi::lang::Kernel&)
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::Program::compile(taichi::lang::Kernel&)
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::Kernel::compile()
/tmp/taichi-qe4n7sb9/taichi_core.so: taichi::lang::Kernel::operator()()
/tmp/taichi-qe4n7sb9/taichi_core.so(+0x793dc9) [0x7ff0ee8b6dc9]
/tmp/taichi-qe4n7sb9/taichi_core.so(+0x6ef07e) [0x7ff0ee81207e]
/usr/bin/python3: PyCFunction_Call
/usr/bin/python3: _PyObject_MakeTpCall
/usr/bin/python3: ) [0x508530]
/usr/bin/python3: PyObject_Call
/usr/bin/python3: ) [0x597e3c]
/usr/bin/python3: _PyObject_MakeTpCall
/usr/bin/python3: _PyEval_EvalFrameDefault
/usr/bin/python3: _PyEval_EvalCodeWithName
/usr/bin/python3: _PyFunction_Vectorcall
/usr/bin/python3: PyObject_Call
/usr/bin/python3: _PyEval_EvalFrameDefault
/usr/bin/python3: _PyEval_EvalCodeWithName
/usr/bin/python3: _PyFunction_Vectorcall
/usr/bin/python3: _PyObject_FastCallDict
/usr/bin/python3: _PyObject_Call_Prepend
/usr/bin/python3: ) [0x597e17]
/usr/bin/python3: PyObject_Call
/usr/bin/python3: _PyEval_EvalFrameDefault
/usr/bin/python3: _PyEval_EvalCodeWithName
/usr/bin/python3: _PyFunction_Vectorcall
/usr/bin/python3: _PyEval_EvalFrameDefault
/usr/bin/python3: _PyFunction_Vectorcall
/usr/bin/python3: _PyEval_EvalFrameDefault
/usr/bin/python3: _PyEval_EvalCodeWithName
/usr/bin/python3: PyEval_EvalCode
/usr/bin/python3: ) [0x676101]
/usr/bin/python3: ) [0x67617f]
/usr/bin/python3: PyRun_FileExFlags
/usr/bin/python3: PyRun_SimpleFileExFlags
/usr/bin/python3: Py_RunMain
/usr/bin/python3: Py_BytesMain
/lib/x86_64-linux-gnu/libc.so.6: __libc_start_main
/usr/bin/python3: _start

Internal Error occurred, check this page for possible solutions:
https://taichi.readthedocs.io/en/stable/install.html#troubleshooting
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import fnmatch, glob, traceback, errno, sys, atexit, locale, imp
Traceback (most recent call last):
  File "/home/yadicao/Desktop/taichi/examples/fractal.py", line 30, in <module>
    gui.set_image(pixels)
  File "/home/yadicao/Desktop/taichi/python/taichi/misc/gui.py", line 91, in set_image
    tensor_to_image(img, self.img)
  File "/home/yadicao/Desktop/taichi/python/taichi/lang/kernel.py", line 528, in wrapped
    return primal(*args, **kwargs)
  File "/home/yadicao/Desktop/taichi/python/taichi/lang/kernel.py", line 460, in __call__
    return self.compiled_functions[key](*args)
  File "/home/yadicao/Desktop/taichi/python/taichi/lang/kernel.py", line 424, in func__
    t_kernel()
RuntimeError: [opengl_api.cpp:link@168] [glsl] error while linking program:
error: Too many compute shader storage blocks (11/8)

archibate · 2020-06-26T04:35:06Z

@Eydcao note that the error message is changed, so you actually encountered another issue, I know why this occur and will fix later.

Eydcao · 2020-06-26T04:52:20Z

Yeah this time at least compute shade appears. So it has reached the glx with the correct llvm version

[Bug] [linux] [opengl] Use RTLD_LOCAL to prevent LLVM symbol conflict…

3aff448

… with GLX

archibate requested review from yuanming-hu, k-ye, zhai-xiao, Eydcao and taichi-gardener June 25, 2020 10:36

k-ye approved these changes Jun 25, 2020

View reviewed changes

yuanming-hu approved these changes Jun 25, 2020

View reviewed changes

archibate added the LGTM label Jun 25, 2020

archibate merged commit 0a434d3 into taichi-dev:master Jun 25, 2020

Eydcao mentioned this pull request Jun 26, 2020

[Bug] Cannot load taichi with opengl 4.6 > 4.3 on Ubuntu 20 (AMD card) #1106

Closed

yuanming-hu mentioned this pull request Jun 28, 2020

[release] v0.6.14 #1346

Merged

archibate mentioned this pull request Jun 29, 2020

[llvm] Further improve LLVM 10 compatibility by using RTLD_DEEPBIND #1355

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] [linux] [opengl] Use RTLD_LOCAL to prevent LLVM symbol conflict with GLX #1326

[Bug] [linux] [opengl] Use RTLD_LOCAL to prevent LLVM symbol conflict with GLX #1326

archibate commented Jun 25, 2020

codecov bot commented Jun 25, 2020 •

edited

Loading

k-ye left a comment

masahi commented Jun 25, 2020

yuanming-hu left a comment

Eydcao commented Jun 26, 2020

Eydcao commented Jun 26, 2020

archibate commented Jun 26, 2020

Eydcao commented Jun 26, 2020 •

edited

Loading

[Bug] [linux] [opengl] Use RTLD_LOCAL to prevent LLVM symbol conflict with GLX #1326

[Bug] [linux] [opengl] Use RTLD_LOCAL to prevent LLVM symbol conflict with GLX #1326

Conversation

archibate commented Jun 25, 2020

codecov bot commented Jun 25, 2020 • edited Loading

Codecov Report

k-ye left a comment

Choose a reason for hiding this comment

masahi commented Jun 25, 2020

yuanming-hu left a comment

Choose a reason for hiding this comment

Eydcao commented Jun 26, 2020

Eydcao commented Jun 26, 2020

archibate commented Jun 26, 2020

Eydcao commented Jun 26, 2020 • edited Loading

codecov bot commented Jun 25, 2020 •

edited

Loading

Eydcao commented Jun 26, 2020 •

edited

Loading